Skip to content

feat(loop): SLM/LLM subtask complexity routing - #218

Closed
raymondginger2018-sudo wants to merge 2 commits into
HKUDS:mainfrom
raymondginger2018-sudo:feat/slm-routing
Closed

raymondginger2018-sudo wants to merge 2 commits into
HKUDS:mainfrom
raymondginger2018-sudo:feat/slm-routing

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor

Summary

Small language models (SLM) fit local/edge/low-cost niches. This PR adds an explicit subtask-class router: high-frequency, low-complexity subtasks (tool-result cleanup, summarization, classification) should ride the SLM path, while deep reasoning stays on the LLM path.

Components

core/loop/slm_routing.py (new, 152 lines)

Pure decision mechanism:

  • route_subtask(task_class, ...) -> RoutingDecision
  • Subtask classes: simple / medium / complex
  • Env-tunable model overrides: DEEPCODE_SLM_MODEL, DEEPCODE_LLM_MODEL
  • DEEPCODE_SLM_ROUTING=0 disables SLM routing (falls back to LLM tier)

core/loop/slm_tasks.py (new, 102 lines)

First consumer: turns the routing decision into a preview-shaping policy for oversized tool results — when cleanup is SLM-grade, the persisted preview is shaped as a clean dense digest; when LLM-grade, raw truncation stays.

Design

  • Zero network, zero async — decision-only, deployment-adaptable
  • Route by task complexity, not by caller identity
  • Clean separation: routing decision vs. execution channel

Files

  • core/loop/slm_routing.py
  • core/loop/slm_tasks.py

Part of GenAI lesson 19 SLM/LLM cost-tier family.

@raymondginger2018-sudo

Copy link
Copy Markdown
Contributor Author

设计说明

问题:目前所有 subtask 都走同一个 LLM,简单任务(字符串处理、文件重命名)和复杂任务(多步推理、代码生成)消耗同样成本与延迟。SLM(小模型)在简单任务上已经足够,但缺乏路由机制。

解法:两个文件——

  • slm_routing.py:核心路由器 route_subtask(),按任务复杂度分 SUBTASK_SIMPLE / MEDIUM / COMPLEX 三档
  • slm_tasks.py:第一个 consumer 示例,展示如何将简单 subtask 交给 SLM

关键设计决策

  • 环境变量覆盖:DEEPCODE_SLM_MODELDEEPCODE_LLM_MODELDEEPCODE_SLM_ROUTING=0(可整体关闭)——不改变默认行为,opt-in
  • 复杂度判定是静态规则(token 数 + 关键词 + 结构特征),不走 LLM 判断,避免"用 LLM 判断要不要用 LLM"的递归成本
  • 与现有 LLM provider 抽象解耦,路由结果只是返回模型名,不碰调用链

测试建议:对三档分别准备典型任务,验证路由结果;再跑 DEEPCODE_SLM_ROUTING=0 验证回退路径

@Zongwei9888

Copy link
Copy Markdown
Collaborator

Thank you for the submission, @raymondginger2018-sudo. Closing this implementation, but noting the direction. Two facts about the current tree: there is no SLM channel (sub-agents inherit the parent model in core/harness/agents/control.py), and the module's "consumer" path never calls a model — it is a regex-based truncation with a different name — so merging it would add configuration without behaviour. The PR description also refers to an existing small-model risk gate that we could not find in the repository. The idea itself (a cheaper model tier for compaction summaries and simple sub-agents) is worth doing; we have recorded it as a planned provider-layer change (a model tier on the phase routing in core/config.py), which is where it belongs. If you would like to take that on, start from model_for_phase / make_llm_provider(phase=...) and include an end-to-end test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants